Like any scientific discipline,
data science involves running experiments typically to explore data or
to build and evaluate predictive models. In Azure Machine Learning and
experiment is a named process. Usually, the running of a script or
a pipeline that can generate metrics and outputs and be tracked in
the Azure Machine Learning workspace. This experiment can be run multiple times
with different data, code or settings. An Azure Machine Learning tracks each run,
enabling you to view run history and compare results for each run. Now, let's have a look at how experiments
are controlled in Azure Machine Learning. When you submit an experiment,
you use its run context to initialize and end the experiment run that is
tracked in Azure Machine Learning. After the experiment run is completed,
you can view the details of the running the experiments tab in
Azure Machine Learning studio. Experiments are most useful
when they produce metrics and outputs that can be tracked across runs. Every experiment generates log files that
include the messages that will be written to the terminal during
interactive execution. This enables you to use simple print
statements to write messages to the log. However, if you want to record named
metrics for comparison across runs, you can do so by using the run object
which provides a range of logging functions specifically for this purpose. These include, log,
recorded single named value, log_list, recorded named list of values. Log_row, record a row with
multiple columns, log_table, recorded dictionary as a table and
log_image, recorded image file or a plot. For more information about logging
metrics during experiment runs, see Monitor Azure ML experiment runs and
metrics on the Microsoft website. You can find a link to this resource from
the additional readings at the end of this lesson. For example, you can use a code that records the number
of observations records in a CSV file. You can view the metrics logged by an
experiment run in Azure Machine Learning studio or by using the RunDetails
widget in a notebook. You can also retrieve the metrics using
the run objects get metrics method. This method returns a json
representation of the metrics. This might then produce
output observations. In addition to logging metrics,
an experiment can generate output files. Often, these are trained machine learning
models but you can save any sort of file and make it available as
an output of your experiment run. The output files of an experiment
are saved in its outputs folder. The technique you use to add files
to the outputs of an experiment, depends on how you're
running the experiment. The examples mentioned here so far control the experiment life
cycle in linning your code. When taking this approach, you can upload
local files to the runs outputs folder by using the run objects upload file
method in your experiment code. When running an experiment
in a remote compute context, which we'll discuss later in this course. Any files written to the output folder
in the compute context are automatically uploaded to the runs outputs
folder when the run completes. Whichever approach you use
to run your experiments, you can retrieve a list of output
files from the run object, which would produce an output,
showing output/sample.csv. You can run an experiment in line using
the start_logging method of the experiment object, but it's more common to
encapsulate the experiment logic in a script and
run the script as an experiment. The script can be run in
any valid compute context, making this a more flexible solution
from running experiments at scale. An experiment script is just a python code
file that contains the code you want to run in the experiment. To access the experiment run context,
which is needed to log metrics, the script must import the azureml.core.runclass and
code its get context method. The script can then use the run context
to log metrics, upload files and complete the experiment. To run a script as an experiment, you must
define a script configuration that defines the script to be run and
the python environment in which to run it. This is implemented by using
a ScriptRunConfig object. For example, the code could be used to
run an experiment based on a script in the experiment files folder, which must
also contain any files used by the script, such as the data.csv file in
the previous script code example. It's important to note and implicitly
created run configuration object, defines the python environment for
the experiment, including the packages
available to the script. Also, if your script depends on
packages that are not included in the default environment. You must associate the ScriptRunConfig
with an environment object that makes use of a CondaDependencies object to
specify the python packages required. Runtime environments are discussed
in more detail later in this course.